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ABSTRACT 



This thesis records the method, results, and recommendations stemming from 
a sensitivity analysis performed on the output of the Janus(A) combat simulation. The 
analysis was based on a previous National Training Center (NTC) force-on-force 
(MILES instrumented) battle that was qualified as a Janus(A) scenario by TRAC- 
Monterey. The method involved four modifications to the qualified scenario and the 
use of a factorial experimental design. The factorial design was used to determine the 
presence of relationships between the levels of four main effects (battle parameters) 
and two response variables (Measures Of Performance). The thesis concluded that 
Janus(A) output can demonstrate a statistically significant sensitivity, and that 
Janus(A) sensitivity is a function of the chosen response variable and the levels of each 
main effect. Interactions among three of the four battle parameters were also 
statistically significant. It is recommended that examination of sensitivity analyses 
of Janus(A) be continued and a post-NTC rotation training similar to the one described 
in the thesis be pursued. 
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I. 



INTRODUCTION 



A. PURPOSE 

This thesis concerns the potential benefits of linking the 
Janus (A) combat simulation with training conducted at the 
National Training Center (NTC) . The primary objectives of 
this thesis are: 

1. to identify potential training benefits gained by using 
Janus (A) to enhance post - NTC training; 

2. to examine the sensitivity of the Janus (A) simulation to 
changes in a battle; 

3. to develop a methodology for using Janus (A) to enhance 
a tactical commander's ability to determine training areas 
that require emphasis, and; 

4. to apply work previously conducted by personnel at 
TRADOC Analysis Command-Monterey (TRAC-Monterey ) in 
replicating NTC scenarios with Janus (A). 

A secondary objective of this thesis is to provide information 

that may assist NTC personnel in accomplishing their mission. 

B. SCOPE 

The limited number of NTC laser (MILES instrumented) 
missions that have been successfully modified for execution as 
Janus (A) combat model scenarios constrained the scope of this 
thesis. There is one such scenario available at TRAC- 
Monterey. Currently, it requires approximately 300 hours to 
modify an NTC mission for execution in Janus (A) [Ref. l:p. 
40]. The available NTC battle is a Blue Force (BLUEFOR) 
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battalion conducting a defense-in-sector mission against a 
Motorized Rifle Regiment (MRR) . Some results of this thesis 
are limited to this battle and should not be applied to other 
NTC battles. 

Chapter II of this thesis has a description of the 
Janus (A) combat model and the Army's National Training Center 
(NTC) . The second chapter is not a detailed description of 
Janus (A) or the NTC; rather it contains a discussion of the 
basic components of each. Chapter III is a discussion of the 
process used to examine the sensitivity of the Janus (A) 
simulation. Only changes in the Blue Force's (BLUEFOR) 
operations are considered. The results of this examination 
are in Chapter IV. Chapter V provides the conclusions of 
this thesis and recommendations for future efforts related to 
Janus (A) and the NTC. The Appendix contains a discussion of 
how Janus (A) might be used by tactical units after they have 
trained at the NTC. It includes a method for using Janus (A) 
to enhance the tactical commander's ability to select areas 
that need future training emphasis. 
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II BACKGROUND 



A. DESCRIPTION OF THE JANUS (A) COMBAT MODEL 

1. General 

The Janus (A) combat model is a high resolution 
computer simulation allowing both interactive and systemic 
operations. Interactive operation of Janus (A) allows operator 
interaction and simulates battles in near real time. Systemic 
processing is the ability to execute the Janus (A) model 
without operator interaction. This systemic mode can reduce 
run times to almost 50% of real (or interactive) times. [Ref. 
l:p. 41] 

The model is primarily a two-sided ground force model. 
Janus (A) can simulate both direct fire and indirect fire 
events [Ref. 2:p. 6]. Both direct and indirect fires were 
simulated during the battle considered in this thesis. 
Janus (A) can also simulate rotary and fixed-wing aircraft. 
The scenario considered in this thesis does not include 
simulation of aircraft. 

2. Simulation of Terrain and Environment 

Janus (A) simulates terrain using a system of cells 
arranged to represent a square section of terrain. The 
terrain is modeled as a square measuring 400 cells by 400 



3 



cells [Ref. 2:pp. 35-37]. This is a total of 160,000 cells. 
Characteristics of the terrain such as foliage and elevation 
are constant within each cell. 

For example, if the simulated area measures 40 
kilometers by 40 kilometers then each cell represents a square 
measuring 100 meters by 100 meters. The Janus (A) terrain file 
number 850 represents the NTC maneuver area measuring 40 
kilometers by 4 0 kilometers [Ref. 1] . This is the terrain 
file used throughout this thesis. 

The Janus (A) data base contains up to 16 sets of 
weather conditions. Each set specifies conditions for 
visibility, humidity, temperature and sensor contrast. The 
condition set used in this thesis represents a desert 
environment with good (17 km) visibility, a temperature of 
40°F and low humidity. [Ref. l:p. 25]. 

3 . Simulation of Combat Systems 

Janus (A) can simulate 250 individual units for each 
opposing side. This force level is sufficient to represent a 
Motorized Rifle Regiment (MRR) and an Army Brigade consisting 
of three maneuver battalions. An individual unit can 
represent any one man, single vehicle, artillery piece, or 
similar entity. A unit can also represent a grouping of 
weapons, e.g., a four-tank platoon. All units of the same 
type have the same characteristics such as weapon type, sensor 
type, movement speeds, etc. The Janus (A) user sets these 
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characteristics. Janus (A) does not permit variations in 
characteristics such as crew proficiency between tanks, for 
example . 

4. Simulation of Direct Fire Events 

One-on-one stochastic attrition processes determine 
the rate at which units are attrited [Ref. 2:p. 6]. 
Comparison of random numbers to a user defined probability 
curve resolves each engagement. The Janus (A) probability-of- 
kill and probability-of-hit curves are functions of range to 
target, weapon system, target type, and target posture (in 
terms of movement and firing angle) . This method of 
engagement resolution constitutes an overall Monte-Carlo 
approach. Units with the same weapon type have the same 
probability-of-hit and probability-of-kill functions. Figure 
1 shows a hypothetical probability-of-kill curve for a 
stationary tank main gun weapon firing at a moving flank 
target. The Janus (A) user sets the ranges and corresponding 
probabilities based on the best classified or unclassified 
empirical data available. 
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Probab 1 I i ty of Kill 




Figure 1 Hypothetical Probability 
of Kill Curve 



5. Post Processing Data 

a. Standard Post-Processing Files 

Janus (A) can generate data files for post 
simulation analysis [Ref. 2:pp. G1-G14]. The files contain 
information such as number of artillery missions fired, 
attrition results, and engagement ranges. Janus (A) constructs 
similar files for each side. These files have a standard 
format, but are not readily imported into the statistical 
software package (STATGRAPHICS) used in this thesis. This 
import limitation is due to the repetition of header 
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information throughout the file. Janus (A) output files were 
useful for obtaining a first look at the results of simulated 
battles, but they were not used to provide the data analyzed 
in this thesis. 

b. Specialized Post-Processing Files 

Janus (A) can generate data files for use in the 
Lawrence Livermore National Laboratory Analyst Workstation 
(LLNL AWS) [Ref.lrp. 2]. This analyst workstation provides 
graphical representation of post-run data. Additionally, the 
files created as input for the LLNL AWS are in ASCII format, 
allowing import into the statistical software used in this 
thesis. TRAC-Monterey has also developed a FORTRAN program to 
collect data relating to attrition in the Janus (A) runs. The 
analyses in this thesis are based on these specialized post- 
processing files. Additional FORTRAN programs were written 
to further process information from these files. These 
programs and files are discussed in Chapter III. 

B. DESCRIPTION OF THE NATIONAL TRAINING CENTER (NTC) 

1. Mission 

The Army established the National Training Center 

(NTC) at Fort Irwin, California, in 1981 to: 

(1) provide tough realistic combined arms and service 
joint training in accordance with Airland Battle doctrine 
for brigades and regiments in a mid to high intensity 
environment, while retaining the training feedback and 
analysis focus at Battalion and Task Force level; and. 
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(2) provide a data source for training, doctrine, 
organization, and equipment improvements. [Ref. 3:p. Cl] 

The results of this thesis should assist the NTC in achieving 

its mission by improving a training unit's use of NTC results. 

2 . Terrain 

The Fort Irwin area is characterized by high desert 
terrain with approximately 430,000 acres for tracked vehicle 
movement and 640,000 acres overall. The varied terrain allows 
maneuver across desert valleys, mountains, and defiles. The 
relatively large area allows maneuver that is normally not 
available at a unit's home station. 

3 . Typical Training 

A battalion-size unit deploys to the NTC for 20 days. 
During this period the unit conducts simulated force-on-force 
battles against an NTC-based Opposing Force (OPFOR) . The 
OPFOR consists of two US battalions (one armor, one mechanized 
infantry) configured to replicate the signature of a motorized 
rifle regiment (MRR) [Ref. 3:p. C6] . The OPFOR uses visually 
modified US Army equipment to portray soviet weapon systems. 

Each battalion conducts five to seven of these force- 
on-force battle simulations. The battalion trains under night 
and day conditions. NTC personnel conduct informal After 
Action Reviews (AARs) at the company and battalion levels 
after each force-on-force battle. The example training 
program suggested in the Appendix is linked to information 
unit personnel receive during these AARs. 
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4 . Battle Monitoring System 

The NTC maneuver area has an extensive system to 
monitor and record vehicle movement, direct fire engagements 
and command and control operations [Ref. 3:p. C13]. The data 
acguired by this system are collected. The collected data 
provide the basis for the After Action Reviews (AARs) , Take 
Home Packages (THPs) , and video replays of AARs. Units 
receive copies of the AARs, THPs and video replays. The 
battle monitoring system has limitations in recording all unit 
movement and events. These limitations are one reason it 
takes approximately 300 man-hours to replicate an NTC battle 
in Janus(A) [Ref. l:p. 53]. 

5. Simulation of Direct Fire Events 

Direct Fire events during force-on-force (laser) 
simulations are resolved using the Multiple Integrated Laser 
Effects System (MILES) [Ref. 3:p. CIO]. There are 
approximately 550 MILES equipment sets at the NTC [Ref. 3:p. 
CIO] . The MILES uses an eye-safe laser to simulate the firing 
of a direct fire weapon. Each laser emission is coded to 
represent the firing weapon. If struck by a MILES 's beam, the 
MILES equipment on the target vehicle uses the coded 
information to determine if it has been killed. If the target 
system has been killed, that fact is transmitted to the 
battlefield monitoring system. The battlefield monitoring 
system records killer type, target location, and killer 



9 



location when a firing element successfully engages another 
element. After a successful kill, MILES prohibits the target 
system from engaging other elements. 

6. Post-NTC Training Material 

The Take Home Package (THP) normally includes AAR 
information organized on the basis of the seven Battlefield 
Operating Systems (BOS) [Ref. 3:p. Dl] . The NTC provides THPs 
for company and battalion levels. The THPs include 
information on, 

1. the attrition of OPFOR attributed to each BLUEFOR 
company ; 

2. the ammunition expenditure by weapon system; 

3. the rounds per kill for each direct fire weapon; 

4. the total attrition of each side; and, 

5. a record of graphical control measures (video). 

The packet does not recommend a methodology to improve on 
training related performance deficiencies demonstrated by the 
unit at the NTC. The example methodology discussed in the 
Appendix is designed to aid units in identifying such 
deficiencies and in selecting corrective actions. 

C. TECHNICAL REPORT: COMPARISON OF THE JANUS (A) COMBAT MODEL 
TO NATIONAL TRAINING CENTER (NTC) BATTLE DATA 
1. Purpose and Scope 

As part of an ongoing Army Regulation 5-5 Study called 
Janus and NTC Comparison, TRAC-Monterey has completed a draft 
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technical report, Comparison of The Janus (A) Combat Model to 
National Training Center ^NTC^ Battle Data / TRAC-RDM-TR-191 
[Ref 1] . The draft report published in March 1991 documented 
phase II of the Janus and NTC comparison and had three 
objectives : 

1. to enhance documentation of the process (process also 
referred to as qualification) used to modify an archived NTC 
battle for execution in the Janus (A) combat simulation; 

2 . to conduct data comparison between the actual NTC battle 
and the replicated Janus (A) scenario; and, 

3. to identify limitations discovered in the qualification 
process [Ref. l:p. 1]. 

The report included information on the qualification of a 
single defense-in-sector battle. This thesis is an extension 
of these efforts by TRAC. Much of the research and all the 
actual simulation were conducted at the TRAC-Monterey 
facility. The results in this thesis may help TRAC's future 
efforts in phase III of the Janus and NTC Comparison Study. 

2 . Results and Recommendations 
a. Results 

The technical report documented the qualification 
procedure applied to an NTC battle. It identified the steps 
necessary to adapt NTC digital data for use in Janus (A). 
Direct fire intensity and attrition were used to compare the 
Janus (A) battle with the original NTC battle. The report 
concluded that overall BLUEFOR volume of fire was slightly 
lower in Janus (A) than at the NTC. The report also concluded 
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that a comparison of temporal and spatial direct fire data 
indicated differences in most BLUEFOR companies. Those 
companies or crews identified at the NTC as having poor 
command and control or below average crew performance had a 
statistically significant lower volume-of-f ire than 
corresponding replicates in Janus(A). [Ref. l:pp. 41-47] 

Attrition summary, and temporal-spatial data 
indicated that the BLUEFOR and OPFOR abilities to kill are 
significantly greater in Janus (A) than at the NTC. The report 
attributed much of this difference to the greater 
single-shot-kill probabilities (SSKPs) used in Janus (A) during 
this simulation. Because the Janus (A) probability-of-kill for 
each weapon were modified to reflect NTC MILES relationships, 
the report judged probability-of-hit to be the basis for SSKP 
differences. This is reasonable because Janus (A) uses SSKP = 
probability-of-hit x probability-of-kill (SSKP = PH x PK) . 
The report concluded that the probability of hit data used 
within Janus (A) are optimistic as compared to those 
demonstrated in this NTC battle [Ref. l:pp. 47-52]. 
b. Recommendations 

The technical report states four recommendations. 
First, that tactical commanders use this ability to replicate 
NTC battles within Janus (A) to enhance their training after an 
NTC rotation. This thesis is one result of this 

recommendation. Second, that the NTC data collection process 
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should be changed to increase the accuracy of collected data 
and to reduce the time required to qualify NTC missions in 
Janus (A). Third, that the qualification process should be 
automated to the greatest possible extent. The report states 
that automation will reduce the estimated 300 man-hour 
qualification process. Fourth, that improvements should be 
made in Janus (A) so that algorithms portray more accurately 
variations in company or crew performance. [Ref. l:pp. 53-60] 

D. US GENERAL ACCOUNTING OFFICE REPORT, NTC POTENTIAL 

In the summer of 1986 the General Accounting Office (GAO) 
completed a report to the Secretary of the Army entitled ARMY 
TRAINING National Training Center's Training Potential Has Not 
Been Realized [Ref. 4]. The GAO sought to determine if the 
Army was using the information collected from NTC exercises to 
analyze deficiencies in unit performance, to determine their 
causes and initiate solutions. The study concluded that the 
Army had not used the objective data for overall assessment of 
organizations or weapon systems. The GAO determined that the 
Army had not identified causes of Army-wide deficiencies nor 
had the Army initiated solutions. The study recommended that 
the Army address the development of performance measures, task 
lists and analysis methodologies. The study identified 
integrating home station training with NTC training as an 
Army-identified objective that would help to solve the data 
analysis problem. This thesis provides an example of how a 
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unit might integrate post-NTC training and Janus (A). [Ref. 
4:pp. 2-4] 



14 



Ill PROCESS FOR EVALUATING THE SENSITIVITY OF JANUS (A) 



A. GENERAL 

The objective of the five-part process for evaluating the 
sensitivity of Janus (A) is to investigate how sensitive the 
Janus (A) output is to changes in the BLUEFOR operation. 
Figure 2 depicts this five part process. 



Conduct of the Experiment 

/ 

Selection of Measures Of Performance 

/ 

Selection of an Experimental Design 

/ 

Selection of BLUEFOR Parameters to Vary 

/ 

Selection of A Battle for Analysis 

Figure 2 Five Part Process for Evaluating Janus (A) 
Sensitivity 



This process required the selection of at least one 
Measure of Performance (a response variable sensitive to 
changes in BLUEFOR operations) . This Measure Of Performance 
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(MOP) provides information on how the simulation reacts to 
BLUEFOR changes. Since the selected response variable 
indicates how well the BLUEFOR accomplished its mission, it is 
a measure of the BLUEFOR 's performance. Because of this 
relationship to the BLUEFOR operation, the response variable 
was referred to as a Measure Of Performance (MOP) . The MOP is 
described in detail in section III E. 

Selected areas of the BLUEFOR operation are modified by 
changing certain battle parameters. The parameters are 
equivalent to the main effects (factors) of an experimental 
design. A discussion of the parameters used in this thesis is 
given in section III C. The process involves exploring the 
relationships between the parameters (main effects) and the 
MOP (response variable) with the 2*^ factorial design described 
in section III D. 

B. PART I. SELECTION OF A BATTLE FOR ANALYSIS 

1 . General 

The selected battle was the NTC Defense in Sector 
(DIS) battle discussed in the TRAC-Monterey Technical Report 
[Ref. 1] . There are two reasons for selecting this battle. 
First, the fact that this mission was previously simulated at 
the NTC lends credibility to this DIS scenario. The 
development of a similar, but original Janus (A) scenario would 
require the justification of force disposition, enemy course 
of action, etc. Selecting this DIS battle removes the need 
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for such justification. Second, the qualification of another 
NTC battle would currently require an additional 300 man- 
hours, which would significantly delay the project's 
completion . 

2. Description of the Selected Battle 
a. Duration and Critical Events 

The actual NTC battle occurred in fiscal 1988 from 
approximately 12:00 p.m. until 4:40 a.m. This period includes 
the start of the DIS mission until the time when the BLUEFOR 
received an order to change its mission. The total battle 
time was approximately 300 minutes [Ref. l:p. 14]. 

The TRAC-Monterey technical report selected a 160 
minute segment of the NTC battle for simulation in Janus (A) 
[Ref. l:p. 14]. The period covered is 2:00 a.m. through 4:40 
a.m. Three factors determined this 160 minute time segment. 
First, the Janus (A) simulation available at TRAC-Monterey 
currently limits a single NTC simulation to approximately 245 
minutes. This prohibits simulation of the complete NTC battle 
in Janus (A) . Second, the most critical events occurred during 
this 160 minute period. Table I lists critical NTC battle 
events [Ref. l:p. 13]. 
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Table I CRITICAL EVENTS, NTC BATTLE 



TIME - 24 
Hour clock 


Event 

t 


0200 


OPFOR Forward Detachment (Fwd Det) crosses 
Phase Line (PL) Wendy 


0225 


OPFOR Fwd Det vicinity TRP 11. 

Lead Elements of MRR vicinity 37 grid line 


0245 


MRR vicinity PL Whitley 

Team (Tm) F engages Fwd Det from BP 21A 


0300 


MRR vicinity 48 98 grid square. 

Tm F and D Company (Co) engage Fwd Det in 
Engagement Area (EA) Cuda. 


0330 


D company lost 5 of 10 tanks, and one 
Improved TOW Vehicle (ITV) . 

Fwd Det is combat ineffective. 

MRR vicinity Target Reference Point (TRP) 
11. 


0400 


Lead elements of MRR vicinity EA Piranha. 
D Co(-), Tm F and B Co are combat 
ineffective. 


0430 


MRR crosses PL Abercrombie with 




approximately 60% strength. 


0440 


BLUEFOR receives change of mission 



The third factor supporting selection of this time segment was 
the NTC digital data. The data base was available for the 
period 2:00 a.m. to 4:40 a.m. [Ref. l:p. 14]. 
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b. Disposition of Forces 



Table II lists the critical direct fire systems 

present at the beginning of the battle [Ref 5] . 

Table II CRITICAL DIRECT FIRE 
SYSTEMS 



Critical BLUEFOR Direct Fire Systems 



SYSTEM 


UNITS 


Team 

A 


Team B 


Team F 


Team 

D 


Task 

Force 

Control 


Total 


Tank 


I® 


13 


4 


10 


2 


39 


M2/M3 

Bradley 


7 


0 


7 


0 


4 


18 


ITV 


0 


0 




5 


0 


5 


ARC 


3 


• 


5 


3 


§ 


18 


Critical OPFOR Direct Fire Systems 


SYSTEM 


UNITS 


Fwd 

Det 


1st 

Echelon 


2nd 

Echelon 


Regimental 

Control 


Total 


Tank 


13 


26 


16 


1 


56 


BMP 


21 


46 


32 


9 


108 


BRDM 


1 


1 


0 


5 


7 


MTLB 


16 


0 


0 


0 
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Figures 3 through 8 indicate the disposition of 
both OPFOR and BLUEFOR elements at the beginning of the 
Janus (A) simulation. Figure 3 depicts the overall BLUEFOR. 
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Figure three shows graphics that the BLUEFOR commander used to 
orient his forces to the South-West along the anticipated 
enemy avenue of approach. Also shown are the vehicle 
positions of the BLUEFOR Companies and Teams. The BLUEFOR 
reconnaissance element (Team Scout) was located along Phase 
Line (PL) Victoria. 
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Figures three, four, five, and six show individual 
company positions. These figures are from the qualified 
Janus (A) battle. The Janus (A) simulation uses numerical 
designations for companies and teams. In this scenario Team 
A elements are designated by "2", Team B by "4", D Co(-) by 
"5", and Team F by "3". Team A occupied Battle Position (BP) 
BP 22 at the start of the battle. Team A was to orient fires 
into engagement area (EA) Shark. 




Figure 4 Location of BLUEFOR Team A 
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Team B occupied in the area of BP 24. Their mission 
was to destroy forces in EA Piranha and on order act as the 
BLUEFOR reserve. 




Figure 5 Location of BLUEFOR Team B 
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D Company minus (D Co (-) ) established positions 
forward of BP 25. The forward positions were an attempt to 
deceive the OPFOR. The deception was designed to persuade the 
OPFOR into believing the BLUEFOR was farther to the West than 
its actual Battle Positions. 
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Team F was located West of Battle Position 21; near 
an area referred to as Red Pass. Team F was to orient its 
fires into EA Cuda and on order react to the North to destroy 
OPFOR elements in EA Piranha. 
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The OPFOR positioned themselves to the West of the 
initial BLUEFOR positions. Figure eight depicts the OPFOR at 
the start of the Janus (A) simulation. At the beginning of the 
160.0 minute segment some OPFOR reconnaissance elements had 




Figure 8 Location of OPFOR Elements 
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c. Blue Force: Defender 



(1) Mission The THP contains the BLUEFOR 
commander's mission statement. The mission statement read in 
part, "Task Force ... defends in sector from PL VICTORIA to PL 

GERALD [Northweast of PL Lawton] NLT 242400 [ ] 88 to 

destroy enemy forces and allow no penetration of PL LAWTON." 
[Ref. 5] 

( 2 ) BLUEFOR Concept of the Operation Additional 
guidance on how the commander wanted to accomplish the BLUEFOR 
mission is included in the commander's concept of the 
operation. The concept had two phases. The first phase was an 
aggressive counter-reconnaissance battle. This counter- 
reconnaissance battle required destruction of the OPFOR's 
reconnaissance assets to include the Forward Detachment and 
Deep Reconnaissance Teams (DRTs) . The second phase of the 
BLUEFOR operation was to destroy the remaining OPFOR elements 
in Engagement Areas (EAs) Shark, Piranha and Cuda. [Ref. 5] 

(3) BLUEFOR Task Organization Table III shows the 
BLUEFOR task organization. The Janus (A) simulation included 
three artillery batteries not listed in this task 
organization. These batteries fired missions in support of 
the BLUEFOR and its parent Brigade at the NTC. These missions 
are conducted in the Janus (A) scenario as well. The parameter 
changes used in this thesis did not involve the BLUEFOR 
artillery. 
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Table III BLUEFOR TASK 
ORGANIZATION 



Tin Scout 
Scout Platoon 
2 X Stinger Sect. 

B Co 

1 PLT / B Armor 

2 PLT / B Armor 

3 PLT / B Armor 

1 Stinger Section 



Tm A 

1 PLT / A Armor 

2 PLT / A Armor 
2 PLT / A Mech 

2 PLT / B Mech 
D Co f-) 

2 PLT / D Armor 

3 PLT / D Armor 

1 PLT / E Anti-Tank 
Vulcan PLT (DS) 



Tm F 

1 PLT / A Mech 
3 PLT / A Mech 
3 PLT / A Armor 
1 Stinger Sect. 
Task Force Control 
HQ Tank Section 
A / EN (-) 

1 PLT / A / EN 
Hvy Mortar PLT 



d. Opposing Force: Attacker 

(1) OPFOR Mission and Concept of the Operation The 
video portion of the THP included comments by the OPFOR 
commander. The OPFOR commander stated the following mission, 
"To conduct a Regiiriental attack from positions in contact at 

250230 [ ] 88" [Ref. 5]. The purpose of this attack was 

to penetrate the BLUEFOR forward positions and destroy the 
majority of the BLUEFOR 's combat power. The OPFOR was also to 
preserve enough of its own combat power to repel a BLUEFOR 
counter attack. The OPFOR commander inserted Deep 
Reconnaissance Teams (DRTs) and Regimental Reconnaissance 
Teams (RRTs) to collect intelligence on BLUEFOR activities. 
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The OPFOR commander then used this intelligence to develop his 
scheme of maneuver. [Ref. 5] 

( 2 ) OPFOR Task Organization Table IV lists the 
OPFOR combat units. Not shown are the OPFOR combat support 
(CS) and combat service support (CSS) units. 



Table IV OPFOR TASK ORGANIZATION 

Fwd Pet 1st Ech** 2nd Ech Regimental Control 

2d MRB* 3d MRB 1st MRB HQ Elements 

* Motorized Rifle Battalion (MRB) 

** Echelon (Ech) 



e. NTC Battle Results 

Two of the quantitative measures NTC personnel use 
to determine the results of a battle are force attrition and 
the average number of BLUEFOR rounds fired for each OPFOR kill 
(rounds-per-kill) . The NTC personnel tabulate attrition 
results for both the OPFOR and BLUEFOR. The AARs include the 
rounds-per-kill information for BLUEFOR only. 

The attrition information is used to determine how 
many of the OPFOR losses are attributed to BLUEFOR systems. 
This value is then used to assess how well the BLUEFOR 
performed the mission of destroying the OPFOR. Table V shows 
the attrition results for selected direct fire weapon systems. 
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The NTC personnel recorded a total 61 OPFOR vehicles and 47 

BLUEFOR vehicles destroyed during the battle [Ref. 5]. 

Table V SELECTED NTC VEHICLE 
LOSSES 



NTC Battle Losses for Critical Systems 


BLUEFOR 


OPFOR 


System 


Quantity Lost 


System 


Quantity Lost 


Tank 


29 


Tank 


18 


M2/M3 

Bradley 


5 


BMP 


33 


ITV 


4 


BRDM 


5 


APC 


5 


MTLB 


2 


TOTAL 


43 


TOTAL 


58 



The rounds-per-kill data are used to assess how 
efficient the BLUEFOR was at destroying OPFOR vehicles. Table 
VI shows the rounds-per-kill results of this NTC battle. For 
comparison, NTC personnel stated that the average 
rounds-per-kill for BLUEFOR tanks (11.7) was greater than the 
overall average for tanks at the NTC [Ref. 5]. 
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Table VI ROUNDS EXPENDED AT THE 
NTC 



BLUEFOR: Rounds Expended for i 


Selected Systems at the NTC 


BLUEFOR Unit 


Tank Main Gun 


TOW Missile 


M2/M3 Main Gun 


Tm A 


^ 26 


2 


625 


B Co 


48 


N/A 


N/A 


Tra F 


81 


2 


900 


D Co (-) 


115 


49 


N/A 


TOTAL Fired 
/OPFOR kills 


304 / 26 


51/2 


1525 / 6 



C. PART II. SELECTION OF THE BLUEFOR PARAMETERS TO BE VARIED 
IN JANUS (A) 

1. General 

Four battle parameters were selected; 

1. the position of certain vehicles (vehicle positions) ; 

2. the engineer survivability preparation of vehicle 
fighting positions (survivability positions) ; 

3. the volume of fire for direct fire weapon systems 
(volume-of-f ire) , and; 

4. the coordination of reserve force movement (coordination 
of reserve) . 

There are three reasons for selecting these 
parameters. First, comments in the THP indicated certain 
deficiencies in these factors during the actual NTC battle. 
The engineer failed to emplace all the survivability positions 
required by the BLUEFOR operations order (OPORD) . The 
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majority of these survivability positions were to be located 
in the three Battle Positions (BPs) numbered BP 22, BP 25 and 
BP 21. According to the THP, the reserve force may have moved 
too late to significantly influence the battle, and the actual 
move was unorganized. The THP video AAR record of rounds 
fired showed that 24 vehicles did not fire during the battle. 
The personnel who conducted the AAR attributed this lack of 
fire to poor vehicle positioning and poor crew gunnery skills. 
[Ref. 5] 

Second, changes in a selected parameter should not 
require changes in the OPFOR scheme of maneuver. The intent 
is to simulate the battle with changes in the BLUEFOR while 
the OPFOR 's actions remain the same. This requires that the 
changes in the BLUEFOR operation be hard to detect by the 
OPFOR. Radical changes in the disposition of the BLUEFOR 
might very well provide additional information to OPFOR Deep 
Reconnaissance Teams and Regimental Reconnaissance Teams. 
This information would be in addition to that already gathered 
by the teams. With additional information the OPFOR commander 
could decide to change his scheme of maneuver. 

Third, these four parameters have an assumed 
relationship to how well the BLUEFOR accomplished its mission. 
The mission entails destruction of the enemy and an inherent 
need to preserve the friendly forces. Survivability positions 
help to preserve the BLUEFOR. Changes in volume-of-f ire, 
coordination of the reserve and vehicle positions support the 
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destruction of the OPFOR, as well as attempt to preserve the 
BLUEFOR. 

Only one change is applied to each parameter. This 
equates to each parameter having two conditions or levels. 
One condition is the original state of the parameter in the 
qualified NTC scenario. The other condition or level is the 
result of a change to the BLUEFOR operation. For example, 
the number of survivability positions in the qualified 
scenario corresponds to one level of the survivability 
position parameter. Altering the number of survivability 
positions generates the second level of the survivability 
position parameter. 

2. Vehicle Position Parameter 

The change to vehicle positions was to move selected 
vehicles into BP 22, BP 25 and BP 21 while ensuring each could 
engage into designated engagement areas (EAs) . The support 
for this change comes from two sources: the BLUEFOR 
commander's concept of the operation, and guidance in the Army 
Field Manual FM 71-2 The Tank and Mechanized Infantry 
Battalion Task Force [Ref. 6]. 

Figures 9, 10 and 11 depict the new positions assigned 
to vehicles in Battle Positions (BPs) BP 22, BP 25, BP 21. 
These new vehicle positions constitute changes to the basic 
scenario, but are not necessarily improvements in the BLUEFOR 
operation. 
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Figure nine shows Team A within Battle Position 22. Each 
vehicle can engage into engagement area (EA) Shark. Some 
vehicles had better fields of fire than did other vehicles. 
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Figures ten and eleven show the new positions 



Co(-) and Team F respectively. 




Figure 10 New Location of D Co (-) 



of D 
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During the qualification process many of the D Co(-) 
and Team F vehicles were given instructions to maneuver during 
the battle. The vehicle movements were based on the NTC 
battle. Part of this change in vehicle positions was to alter 
these routes. The alterations ensured the vehicles made 
roughly same movements, but each vehicle's movement began from 
the new vehicle position. 




35 



The commander's concept of the operation was to 
destroy the enemy in EA Shark and EA Piranha, and to prevent 
penetration of OPFOR forces [Ref. 5] . The lines of sight for 
each vehicle in Janus (A) were checked to ensure each vehicle 
could engage OPFOR elements in its primary EA. The guidance 
found in FM 71-2 is the basis for the arrangement of vehicles 
within the BPs [Ref. 6:pp.4~l through 4-102]. This change to 
individual vehicle positions attempts to study the effects 
selecting different positions may have had on the battle 
outcome . 

3. Survivability Position Parameter 

As a result of the qualification process all vehicles 
in the simulation started in a partial defilade status 
[Ref.l:p. 38]. A partial defilade status simulates each 
vehicle in a fighting position that provides a medium amount 
of protection. The change considered in this thesis was to 
deploy vehicles in BP 21, BP 25 and BP 22 in a full defilade 
status. A full defilade status provides the vehicles with the 
maximum protection from enemy direct fire weapons. A full 
defilade position is equivalent to a hull-down fighting 
position. This change required the creation of 42 full 
defilade positions. Comments by the BLUEFOR engineer 
indicate that he attempted to emplace 42 survivability 
positions [Ref. 5]. The exact locations and extent to which 
any one position was used are not in the Take Home Package 
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(THP) . This change was an attempt to study the effects that 
improved engineer support might have had on the battle. 

4 . Volume of Fire Parameter 

The THP indicated that 24 BLUEFOR vehicles did not 
fire during the battle [Ref. 5] . Vehicles that fired zero 
rounds at the NTC were placed in a hold-fire status during the 
qualification process [Ref.l:p. 35]. A hold-fire status in 
Janus (A) prevents the vehicle from firing under any 
circumstance. The modification considered in this thesis is 
to change all those vehicles previously in a hold-fire status 
to a free-fire status. In a free-fire status the weapon may 
fire whenever the engagement criteria of the simulation are 
met. The modification to volume-of-f ire in this thesis 
resulted in all vehicles being deployed in a free-fire status. 
This change is an attempt to determine the effect on the 
battle outcome with all the BLUEFOR vehicles capable of 
engaging targets. 

5. Coordination of the Reserve Parameter 

The BLUEFOR commander designated Team B as the BLUEFOR 
reserve. The changes in coordination of the reserve entailed 
adjusting the movement time and final vehicle positions of 
Team B. The new Team B arrival time to BP 31 was 10-15 
minutes earlier than the arrival time in the basic qualified 
scenario. The final vehicle positions are all nearby Battle 
Position 31 (BP 31) . Team B vehicles were oriented to the 
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West North-West from BP 31. Figure twelve shows the movement 
routes and final vehicle positions near BP 31. 
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D. PART III. SELECTION OF AN EXPERIMENTAL DESIGN 
1 . General 

A 2^ factorial experimental design was selected. This 
design is a subset of the family of factorial experimental 
designs. The name 2^ relates to considering K factors each 
with two levels. In this thesis K = 4. 

The design was selected for several reasons. The 2*^ 
design is particularly useful in the early stages of 
experimental work. The design allows the smallest number of 
treatment combinations with which K factors can be studied 
under a complete factorial arrangement. [Ref. 8:p. 192] 

A 2^ design may provide information on how sensitive 
a response variable is to changes in main effects. This 
characteristic was important in assessing the impact changes 
in parameters had on the battle outcome. This design can 
provide insight into interactions between factors (main 
effects) . Interactions occur when the difference in responses 
between levels of one factor is not the same at all levels of 
the other factors. [Ref. 8;pp. 189-292] 

The final reasons for selecting a 2^ factorial design 
were the values used for the parameters and the possible 
values for the MOP. The parameters could take on two values 
indicating one of two conditions. One value of the parameter 
indicated the original condition. A second value of the 
parameter indicated the changed condition. The values of the 
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MOP were on a ratio scale. A 2*^ design accommodates these 
parameter and MOP values. A discussion of the actual 
parameter values and the ranges for MOP values is given in 
section III F. 

2. Discussion of the 2* Factorial Design 

The 2'’ design is a special case of the general K- 
f actor factorial design. The following equation models the 
relationship between the response variable and the four 
factors : 

= ^ + 'r^+p^. + Yjt+6j+(Tp)^^.+ (TY)ijt-^(T6)^j 

+ (pY)jjt+(P6)j.j+(Y6)j^_j 



where : 

i =1,2; 
j =1,2; 

k =1,2; 

1 =1,2; 
m = 1,2, 3, 4, 5; 

n =5, the number of observations for that combination 
of i, j, k, 1; 

response with corresponding factors at 
their i, j, k, 1 levels; 

^ = overall mean response; 

Tj = effect of the ith level of factor A; 

Pj = effect of the jth level of factor B; 

Yk = effect of the kth level of factor C; 

= effect of the 1th level of factor D; 

and, 

®ijkijn = the random error component. 



In this thesis the factors A, B, C, and D correspond to the 
parameters. Terms of the form (xP)ij indicate the two-way 
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interactions of corresponding parameters (main factors) . The 
values for i, j, k, and 1 are limited to 1 and 2 since this 
design only considers two levels of each factor. As discussed 
in section III F for this thesis, the total number of 
observations is 80. It is assumed that the are 

independent, identically distributed normal with a mean of 
zero and a variance o^. [Ref. 8;pp. 189-210] 

The three and four-way interactions are assumed to be 
negligible and are not included in the model. All four main 
effects are considered to be fixed effects, because the level 
of each is not chosen at random. The four treatment effects 
are defined as deviations from the overall mean, so: 

f^T_^ = 0, f^P =0, f^6j = 0 . 

2-1 j-1 k=l 2=1 

The two-way interactions are also considered to be fixed 
effects. This results in equations relating the interaction 
effects. The following are examples of these equations: 

t (xP),-£ (tP),j = 0, 

i = l J=1 

t t (xy)ijc=0 . 

i=l k=l 

Similar relationships are assumed for the remaining two-way 

interaction teirms. [Ref. 8:pp. 45, 223-228] 
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The model accuracy depends on the assumption that the 
error terms, normally distributed and independent. 
If the errors are from this normal distribution, then a normal 
probability plot of the model's error values should appear 
similar to the plot of a random sample from a normal 
distribution with the same mean and variance. Additionally, 
the behavior of the residuals may be examined by plotting each 
residual against the value predicted by the fitted linear 
model. This plot can be used to check for the presence of a 
relationship between the magnitude of the residual and the 
predicted value. For example, if the residual values increase 
as the predicted values increase then the model may not be 
adequate. [Ref. 8:pp. 201-206] 

E. PART IV. SELECTION OF A MEASURE OF PERFORMANCE (MOP) 

Part IV of the process resulted in the selection of two 
Measures Of Performance (MOPs) . The first MOP (referred to as 
MOPl) is defined: 

^ BLUEFOR ROUNDS FIRED 
OPFOR LOSSES 



Use of the rounds-per-kill measure at the NTC motivates using 
MOPl in this thesis. This MOP may allow comparisons to be 
made between the NTC battle results and the results from the 
Janus (A) scenarios. These comparisons are tempered by the 
results of the TRAC-Monterey Technical Report concerning 
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differences in attrition between Janus (A) and the NTC [Ref. 
1 : pp . 56-58 ] . 

The second MOP is a combined measure of the fractions of 
OPFOR losses and BLUEFOR survivors. This MOP is labeled M0P2 
and is defined by: 

OPFOR LOSSES ^ BLUEFOR SURVIVORS 

MOP2 = + 

INITIAL OPFOR STRENGTH INITIAL BLUEFOR STRENGTH 



M0P2 was selected for its relationship to the BLUEFOR mission 
of destroying OPFOR elements while preserving the BLUEFOR. 
The first term is a measure of how many OPFOR elements were 
destroyed by the BLUEFOR. The second term is a measure of how 
well the BLUEFOR commander was able to preserve his force. 

M0P2 can range in values from zero (0) to two (2) . At the 
start of a battle the value of MOP2 equals 1. The first term 
is 0 and the second term is 1. MOP2 equals 2 if the BLUEFOR 
commander has destroyed 100% of the OPFOR while preserving 
100% of the BLUEFOR. An MOP2 value of 2 is the best the 
BLUEFOR can achieve. If 100% of the OPFOR survive and 100% of 
the BLUEFOR are destroyed, then the value of M0P2 equals 0. 
[Ref. 7:p. 39] 

Values of MOPl were calculated using total BLUEFOR rounds 
fired and OPFOR kills at the conclusion of each battle. 

MOP2 values were determined at the 120.0 minute point of each 
battle and at the conclusion of each battle. Support for 
selecting the 120.0 minute point for MOP2 is given in the 
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TRAC-Monterey Technical Report [Ref. l:p. 13]. That report 
determined that the largest direct fire intensity occurred at 
approximately 120.0 minutes into the Janus (A) simulation. 
This corresponds to the time when the OPFOR 2nd Echelon 
engaged the main BLUEFOR elements near EA Cuda. The 
calculation of M0P2 at the 120.0 minute point in the battle 
and at the conclusion of the battle permits an investigation 
of how MOP2 behaves at a two points in the battle. 

To prevent confusion between the two M0P2 values for each 
Janus (A) battle, the following naming conventions were used: 

1. MOP2A is the M0P2 value determined at 120.0 minutes into 
a Janus (A) battle, and; 

2. M0P2B is the M0P2 value determined at the conclusion of 
a Janus (A) battle. 

These MOP definitions and naming conventions result in three 
MOP values for each Janus (A) battle. 

F. PART V. CONDUCT OF THE EXPERIMENT 
1 . General 

The 2'" factorial design is the cornerstone of the 
experiment. The factorial design was used three times during 
this thesis; once for each MOP. A complete replicate of a 2'' 
design requires 16 cells [Ref. 8:p. 261]. In this thesis the 
cells are groupings of Janus (A) battles. Each cell relates 
only to those Janus (A) battles executed under the same 
parameter levels. Five Janus (A) runs (replicates) were 
conducted for each of the 16 cells. So n = five in the 2 “* 
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design discussed in III D. This requires a total of 5 x 16 
= 80 Janus (A) runs. The time available for this thesis 
limited the number of Janus (A) runs to 80. 

2. Construction of the 16 Janus (A) Scenarios 

Each of the 16 Janus (A) scenarios were prepared at the 
TRAC-Monterey facility. There is one scenario for each of the 
16 cells in the 2'' design. The qualified NTC battle scenario 
in the TRAC-Monterey technical report was one of the 16 
scenarios. This scenario is the base case scenario. The 
Janus (A) files for the base case scenario were copied and then 
modified to generate the other 15 scenarios. 

Each of the 16 scenarios was assigned a unique 
four-digit code. This code represents the level of each 
parameter in the scenario. For example, a code of "1111” 
represents the base case scenario. Each of the four digits is 
limited to the values "1" or "2". A value of "1" indicates 
that the parameter level was not changed from the base case. 
A value of "2" indicates that the corresponding parameter has 
been changed. The order of the parameters represented in the 
four-digit code is survivability positions (SP) , volume of 
fire (VF) , vehicle positions (VP) and coordination of the 
reserve (CR) . Table VII shows the four-digit codes with a 
brief explanation of changes. 
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Table VII CODES FOR THE 16 
JANUS (A) SCENARIOS 



TREATMENTS 
SP VF VP CR 


JANUS (A) 
SCENARIO 


EXPLANATION OF CHANGE CODE 


1 


1 


1 


1 


760 


Base Case 


1 


1 


1 


2 


768 


Coordination of Reserve 


1 


1 


2 


1 


764 


Vehicle Positions for Tm A, 
D Co(-) , Tm F 


1 


1 


2 


2 


772 


Vehicle Positions and 
Coordination of Reserve 


1 


2 


1 


1 


76i 


Volume of Fire 


1 


2 


1 


2 


769 


Volume of Fire, Coordination 
of Reserve 


1 


2 


2 


1 


769 


Volume of Fire, Vehicle 
Positions 


1 


2 


2 


2 


775 


Vehicle Positions, Volume of 
Fire, Coordination of 
Reserve 


2 


1 


1 


1 


762 


Survivability Positions 


2 


1 


1 


2 


771 


Survivability Positions, 
Coordination of Reserve 


2 


1 


2 


1 


766 


Survivability, Vehicle 
Positions 


2 


1 


2 


2 


773 


Survivability, Vehicle 
Positions, Coordination of 
Reserve 


2 


2 


1 


1 


764 


Survivability, Volume of 
Fire 


2 


2 


1 


2 


770 


Survivability, Volume of 
Fire, Coordination of 
Reserve 


2 


2 


2 


1 


767 


Survivability, Vehicle Pos, 
Volume of Fire 


2 


2 

1 


2 

1 


2 

1 


774 


Survivability, Vehicle 
Positions, Volume of Fire, 
Coordination of Reserve 
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3. Systemic Operation of Janus (A) 

The 80 Janus (A) battle scenarios were conducted using 
the systemic processing feature in Janus (A). The steps used 
to execute each run were the same as those used in the TRAC- 
Monterey Technical Report [Ref. l:p. 41]. This includes 
reducing possible correlation between runs by randomly 
selecting the initial random number seed for each scenario. 

4. Processing Data from Janus (A) Systemic Runs 
FORTRAN programs were written to process the post- 

Janus (A) simulation data. These FORTRAN programs generated 
two types of output files. One file contained information 
related to a particular Janus (A) runs. The second file 
contained information pertaining to each of the 80 battles. 

Three files provide the input for these FORTRAN 
programs. Two of the input files are generated from the 
Janus (A) simulation data conversion processor. These two 
files are data files normally used as input to the LLNL AWS . 
These files contain information on kills and direct fire 
events. The third input file was the data file generated by 
the FORTRAN program developed at TRAC-Monterey . 

5. Preparation of Post-Janus (A) Battle Data for Analysis 
The STATGRAPHICS statistical software package was used 

in the conduct of this thesis [Ref. 9]. The files generated 
by the FORTRAN programs were in a format easily imported into 
STATGRAPHICS . 
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Four variables were created in the STATGRAPHICS work 
space. There was a variable for each of the parameters varied 
in the BLUEFOR. Each variable contained 80 values. Each of 
the 80 values represented the level code for one of the 
Janus (A) runs, so the variables were one-dimensional arrays 
consisting of ones and twos. The variables were used to 
represent the main effect values for the 2'' factorial design 
and to assist the creation of plots in STATGRAPHICS. 
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IV RESULTS OF THE JANUS (A) SENSITIVITY ANALYSIS 



A. GENERAL 

The results of the sensitivity analysis are discussed 
under two parts. The first part concerns results attributed 
to an exploratory data analysis. This exploratory analysis 
pertained to all 80 values generated for each MOP. The 
creation and analysis of notched-boxplots are major portions 
of this exploratory data analysis. 

The second part concerns results credited to the 2'* 
factorial design discussed in Chapter III. These results 
include a discussion of how changes in the parameters (main 
effects) affected MOP values and a discussion of how 
interactions between parameters affected the MOP. 

B. RESULTS FROM THE EXPLORATORY DATA ANALYSIS 

1. General 

The exploratory data analysis did not provide 
information to dispute the validity of the 2'* factorial 
design. Also, the exploratory analysis indicated that MOPl 
was less sensitive to changes in the parameters than was M0P2 . 

The exploratory data analysis focused in the following 
two areas for each MOP. One area was in the appreciation of 
variation in the MOP values. This appreciation of the data 
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came mainly from the information contained in notched- 
boxplots. A notched-boxplot is a graphical display of one 
dimensional data. The notched-boxplot shows a measure of 
location (median) , two measures of dispersion (interquartile 
range and a confidence interval for the median) and an 
indication of skewness (whiskers and the presence of 
outliers) . [Ref. 10:pp. 336-338] 

The second area of the exploratory analysis concerned 
the assumed homogeneity of variance required by the 2“* design. 
This assumption warranted consideration in the exploratory 
data analysis since the factorial model is sensitive to 
deviations from homogeneity of variance. The test for 
homogeneity in variance involves the test statistic C [Ref. 
lltpp. 61-62, 537]. The test statistic is defined as C: 

MAX.d] 

C = 

g 8? ' 

i=l 

where 6^^, i=l,2, . . . , 15, 16, are the sample variances. 

2 . MOPl 

Recall that MOPl is defined as: 



MOPl 



BLUEFOR ROUNDS FIRED 
OPFOR LOSSES 



The notched-boxplots for the values of MOPl indicated that 
MOPl was not responsive to changes in the four parameters. 
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This indication comes from the overlapping of many of the 
notches. Since the notches respresent 95% confidence 
intervals for the medians, these notched-boxplots indicate 
overlapping of the confidence intervals for medians. 

Figure 13 contains 16 notched-boxplots. There is one 
notched-boxplot for each of the 16 cells. The boxplots are 
for the five MOPl values corresponding to one combination of 
parameter levels. The boxplots are labeled by their 
respective four-digit codes. 



M0P1 ■ BLUEFOR ROUNDS / OPFOR LOSSES 
AT THE CONCLUSION OF THE BATTLE 




FOUR-DIGIT CODE FOR PARAMETER LEVELS 



Figure 13 Multiple Notched-Boxplots for MOPl 
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The median values of MOPl range from 2.458 for code 
2222 (changes applied to all four parameters) to a high of 
2.993 for cell 1211 (changes to volume-of-fire only). These 
values equate to roughly 2.5 and 3.0 rounds-per-kill . One 
reason for this small spread in median MOPl values may be the 
processing of direct-fire engagements in Janus (A). The 
following is a discussion to support this view. 

Consider one engagement between one firing vehicle and 
one target. During this engagement the firing vehicle is a 
BLUEFOR vehicle in a stationary position. Again, the Janus (A) 
Single-Shot-Kill-Probability is; 

SSKP = Pi (K ill I Range, Weapon, Target Posture) x 
Pi {Hit I Range, Weapon, Filer and Target Posture) . 

Let us assume that this engagement takes place over a 
relatively short time span. A span of time short enough that 
the range to target has not changed more than a few hundred 
meters. In this scenario the average off-road movement rate 
was 17 kilometers-per-hour [Ref. l:p. 25]. It is not 
unreasonable for an engagement to be completed without the 
range changing more than a few hundred meters. 

Referring to the hypothetical kill curve in Chapter 
II, the probability-of-kill does not change greatly for a few 
hundred meter change in range, unless of course the change in 
range takes place across the minimum or maximum ranges. This 
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extreme case at the range limits is not considered. The rate 
of change for probability-of-kill curves are similar to those 
for probability-of-hit curves. 

In this engagement sequence the firing vehicle fires 
until it registers a kill. This is approximately equivalent 
to a series of independent Bernoulli trials where the firing 
system stops engaging after the first success [Ref. 10:p. 34]. 
The probability of a success in each of these trials equals 
the SSKP value. 

None of the parameter changes involved changing the 
probability-of-hit or probability-of-kill for any weapon. 
This "insulation" of the SSKPs from parameter changes may 
account for the lack of response in MOPl. 

There was no indication of heterogeneity of variance 
among the 16 cells for MOPl. Results from the test for 
homogeneity of variance failed to support rejection of the 
hypothesis that the cell variances are equal. Table VII 
contains a summary of the test [Ref. ll:p. 537]. 
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Table VIII HOMOGENEITY OF VARIANCE 
TEST, MOPl 



C Test For Homogeneity of Variance. MOPl 


MAX Oi" 


E di" 


C Test Value 


Critical 
Value: 
a = 0.05, 
k = 16, n = 5 


0.32753 


1.53196 


0.2137 


- 0.2419* 



* value for k = 15 



The hypotheses under this test for equal variances were, 

Hq : Oi = 02 = • ' ' = Oi 5 = Oi6 , and 
: some 's are unequal 

where are the variances of the response variable in the 16 

cells. The Hq hypothesis is not rejected if the test 

statistic C is below the critical value. The test statistic 
has an F distribution with k and n-1 degrees of freedom. The 
value of k equals the number of variances and n is the number 
of observations in each cell. For the experiment in this 
thesis k equals 16 and n equals five. [Ref. ll;p. 61] 



3. M0P2A and M0P2B 

The general form of M0P2 is: 

^ OPFOR LOSSES ^ BLUEFOR SURVIVORS 

INITIAL OPFOR STRENGTH INITIAL BLUEFOR STRENGTH 

Figures 14 and 15 contain multiple notched-boxplots for M0P2A 
and M0P2B respectively. In contrast to MOPl, the values for 
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M0P2 appear to respond to changes in the four parameters. The 
combinations of parameter levels that included the volume-of- 
fire parameter (second digit of code) at level 2 generally had 
high M0P2 values. Level 2 for volume-of-f ire reflects all 
BLUEFOR vehicles capable of engaging OPFOR elements. 



M0P2A. M0P2 VALUES CALCULATED AT 
120,0 MINUTES INTO THE BATTLE 




FOUR-DIGIT CODE FOR PARAMETER LEVELS 



Figure 14 Multiple Notched-Boxplots for 
MOP2A 
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The median values for M0P2A range from 0.727 to 1.452. 
These median values are from cell 1122 (changes to vehicle 
positions and coordination of reserve parameters) and cell 
1211 (changes to volume-of-f ire only) , respectively. The 
extremes in variance occurred in cell 1212 (changes to volume- 
of-fire and coordination of the reserve) and cell 1221 
(changes to volume-of-f ire and vehicle positions) . 



M0P2B: M0P2 VALUES CALCULATED AT THE 
CONCLUSION OF THE BATTLE 




FOUR-DIGIT CODE FOR PARAMETER LEVELS 



Figure 15 Multiple Notched-Boxplots for 
M0P2B 



The extreme median values for the cells of M0P2B are 
1.549 for cell 2212 (changes to all parameters except vehicle 
positions) and 0.677 for cell 2122 (change to all parameters 
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except volume-of-f ire) . The range in variance values among the 
three MOPs is largest for M0P2B. The variances ranged from 
0.00118 in cell 1111 (base scenario, no changes) to a high of 
0.07601 in cell 1212 (changes to volume-of-f ire and 
coordination of the reserve) . 

The assumption for homogeneity of variance was not 
rejected by the C test results. The same hypotheses used for 
homogeneity of variance in MOPl were used to test M0P2A and 
M0P2B values. The degrees of freedom were also the same. 
Tables IX and X contain the results of the C test. The 
hypothesis that the ' s are equal was not rejected for either 
MOP2A or MOP2B. 

Table IX HOMOGENEITY OF 
VARIANCE TEST, MOP2A 



C Test For Homogeneity of Variance. MOP2A 



1 






Critical 

Value: 


MAX d." 


E O,^ 


C Test Value 


a = 0.05, 
k = 16, n = 5 


0.0706 


0.3863 


0.18276 


- 0.2419 * 



* Value for k = 15 
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Table X HOMOGENEITY OF VARIANCE 
TEST, MOP2B 



C Test For Homogeneity of Variance. M0P2B 


MAX 6," 


E 6,^ 


C Test Value 


Critical 
Value: 
a = 0.05, 
k = 16, n = E 


0.09359 


0.53142 


0.17611 


- 0.2419 * 



* Value for k = 15 



C. RESULTS OF THE 2* EXPERIMENT 
1 . General 

The results from the factorial experiment are in 
concert with the exploratory data analysis results. None of 
the changes in the four parameters caused a statistically 
significant response in MOPl. One interaction was shown to 
statistically affect MOPl. Both MOP2A and M0P2B demonstrated 
sensitivity to changes in the parameters and the presence of 
interactions . 

The experiment was performed as discussed in Chapter 
III. The statistic used to evaluate the significance of main 
and interaction effects is a ratio of two variance estimates. 
This ratio is the F-ratio. The F-ratio compares mean scjuare 
values associated with treatments and interactions to the 
residual mean square error. 
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An example of this F-ratio is: 



F ratio 



MEAN 

POSITION 

f 

MEAN SQUARE ERROR 
e 



where: 

f = the degrees of freedom for vehicle position (= 1) : 

and, 

e = the degrees of freedom for the residual error (= 69) 
[Ref 10:p. 418] . 

For each main effect (parameter) the degrees of 
freedom equal one. The degrees-of-freedom for each two way 
interaction in this model is one. The degrees-of-freedom for 
the residual error is defined by 

RESIDUAL DEGREES OF FREEDOM = 

TOTAL DEGREES OF FREEDOM 
~y^ parameter degrees of freedom 
interaction degrees Of freedom . 

In this thesis the total degrees-of-freedom is 80-1 = 79. The 
residual mean square error is associated with 79-4-6 = 69 
degrees of freedom. [Ref. 10:p. 417] 
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The linear model discussed in Chapter III was: 



+ (Py),-;,^(P6),.,+ (y6)j,, 



Example hypotheses used in testing each main factor effect in 
this model are of the form: 

: Ti = Tj = 0 ; and. 



Hq is equivalent to stating the parameter associated with t 
has no effect on the mean of values. The hypothesis is 
rejected if the F ratio value is above a critical value. The 
critical value is based upon the same degrees of freedom as 
the F ratio, but is associated with the probability that Hq is 
rejected when it is true. This probability is the type-one 
error, normally denoted by o [Ref. 10:p. 272]. Typical values 
for a are 0.05 and 0.01. In this thesis, a is taken to be 
0.05. The following sections discuss the specific results of 
the 2'' factorial experiment for each MOP. 

2 . MOPl 

The factorial experiment did not demonstrate the 
presence of a relationship between the changes in parameters 
and the mean values of MOPl. This is consistent with the 
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earlier data analysis (Section IV B) . The hypotheses that the 
parameters do not affect the mean MOPl values were not 
rejected. The experiment did indicate the presence of a 
significant interaction that affected MOPl. This interaction 
occurred between the volume-of-f ire parameter and the vehicle 
position parameter. Table XI contains a summary of the 
analysis of variance for MOPl. 
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Table XI SUMMARY OF RESULTS, 
MOPl 



Experimental Results; MOPl - Rounds per Kill 



SOURCE 


SUM OF 
SQUARES 


DEG. 

OF 

FREE. 


MEAN 

SQ. 


F- 

RATIO 


SIG. 

LEVEL 


REJECT 

Ho 


MAIN EFFECTS 


.40579 


4 


. 1014 


1.1 


.3635 




SURVIVABILITY 

POSITIONS 


. 14645 


1 


. 1464 


1.536 


.2113 


NO 


VOLUME OF FIRE 


. 1170 


1 


. 1170 


1.269 


.2638 


NO 


VEHICLE 

POSITIONS 


.1314 


1 


. 1314 


1.425 


.2366 


NO 


COORDINATION OF 
RESERVE 


.0109 


1 


. 0109 


. 118 


.7356 


NO 


TWO-WAY 

INTERACTIONS 


1.064 


6 


. 1773 


1.923 


.0892 




SURVIVABILITY - 
VOLUME OF FIRE 


. 1686 


1 


. 1686 


1.923 


. 1807 


NO 


SURVIVABILITY - 
VEHICLE POSITION 


. 0022 


1 


.0022 


.024 


.8785 


NO 


VOLUME OF FIRE - 
VEHICLE POSITION 


. 6189 


1 


.6189 


6.706 


.0117 


YES 


SURVIVABILITY - 
COORD OF RESERVE 


. 1416 


1 


.1416 


1.536 


.2194 


NO 


VOLUME OF FIRE - 
COORD OF RESERVE 


.0013 


1 


.0013 


.015 


.9055 


NO 


VEHICLE POSITION 
- COORD OF RES 


. 1319 


1 


. 1416 


1.431 


.2357 


NO 


RESIDUAL 


6.362 


69 


.0922 








TOTAL 


7.8319 


79 
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Figure 16 contains a plot of mean values of MOPl 
against levels of the vehicle position parameter. The solid 
line corresponds to the volume-of-f ire parameter at level one, 
and the dashed line is for volume-of-f ire at level two. Each 
mean value is based on the 20 MOP values corresponding to that 
combination of the two parameters. 



Factor Means Plot: M0P1 

Vo I ume of Fire 

““ At Leve I 1 ' At Leve I 2 



M0P1 M0P1 




Levels' Vehicle Position 



Figure 16 Plot of Means for Interaction, 
MOPl 
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Figure 16 suggests that the effect of changing vehicle 
position from level one to level two depends on whether 
volume-of-f ire is at level one or two. With the volume-of- 
fire parameter at level one, changing vehicle position to 
level two is beneficial in terms of MOPl. A low value of MOPl 
(rounds-per-kill) is preferred. 

An investigation of the values comprising MOPl 
provides the basis for one explanation of this interaction. 
More BLUEFOR rounds are fired when volume-of-f ire is at level 
two (a mean rounds fired of 393 verses 301 for level one) . It 
appears that the new vehicle positions allow the BLUEFOR to 
expend fewer rounds while causing similar OPFOR losses. With 
respect to the four mean values in this interaction, the mean 
number of OPFOR losses range from 140 to 155. 

3. M0P2A and M0P2B 

Unlike MOPl, both MOP2A and M0P2B demonstrated 
statistically significant main factor effects. Tables XII and 
XIII contain analysis of variance results for MOP2A and MOP2B, 
respectively . 
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Table XII SUMMARY OF RESULTS, 
MOP2A 



Experimental Results; M0P2A = OPFOR Casualty Rate + BLUEFOR 
Survival Rate, 120.0 minutes into the battle 


SOURCE 


SUM OF 
SQUARES 


DEG. 

OF 

FREE. 


MEAN 

SQ. 


F- 

RATIO 


SIG. 

LEVEL 


REJECT 

Ho 


MAIN EFFECTS 


3 . 0743 


4 


.7685 


32.43 


.0000 




SURVIVABILITY 

POSITIONS 


. 0607 


1 


. 0007 


2 . 565 


. 1138 


NO 


VOLUME OF FIRE 


1.556 


1 


1.556 


65.68 


.0000 


YES 


VEHICLE 

POSITIONS 


. 6d«7 


1 


.8927 


37.67 


. 0000 


YES 


COORDINATION OF 
RESERVE 


.5643 


1 


.5643 


23.81 


.OICC 


YES 


TWO-WAY 

INTERACTIONS 


.7617 


6 


. 1269 


5.357 


. 0001 




SURVIVABILITY - 
VOLUME OF FIRE 


. 0281 


1 


.0281 


1.185 


. §901 


NO 


SURVIVABILITY - 
VEHICLE POSITION 


.2992 


1 


.2992 


12.69 


.0007 


YES 


VOLUME OF FIRE - 
VEHICLE POSITION 


.0068 


1 


.0068 


.287 


.5999 


NO 


SURVIVABILITY - 
COORD OF RESERVE 


. 1383 


1 


. 1383 


5.839 


.0183 


YES 


VOLUME OF FIRE - 
COORD OF RESERVE 


. 0188 


1 


.0188 


.794 


. 3854 


NO 


VEHICLE POSITION 
- COORD OF RES 


.2703 


1 


.2703 


11.41 


.0012 


YES 


RESIDUAL 


1.635 


69 


. 0236 




TOTAL 


5.4711 


79 
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Table XIII SUMMARY OF RESULTS, 
MOP2B 



Experimental Results: MOP2B = OPFOR Casualty Rate + BLUEFOR 

Survival Rate, at the Conclusion of the Battle 



SOURCE 


SUM OF 
SQUARES 


DEG. 

OF 

FREE. 


MEAN 

SQ. 


F- 

RATIO 


SIG. 

LEVEL 


REJECT 

Ho 


MAIN EFFECTS 


3.6417 


4 


.9104 


15.71 


.0000 




SURVIVABILITY 

POSITIONS 


.1353 


1 


.1353 


2.335 


.19S0 


NO 


VOLUME OF FIRE 


1.675 


1 


1.675 


38.90 


.0000 


YES 


VEHICLE 

POSITIONS 


.7059 


1 


.7059 


12.17 


.00T8 


YES 


COORDINATION OF 
RESERVE 


1.125 


1 


1.125 


19.41 


.0000 


YES 


TWO-WAY 

INTERACTIONS 


.5002 


6 


.0833 


1.438 


.2128 




SURVIVABILITY - 
VOLUME OF FIRE 


.0979 


1 


.0878 


1.515 


.2225 


NO 


SURVIVABILITY - 
VEHICLE POSITION 


.2129 


1 


.212C 


3.673 


.0595 


NO 


VOLUME OF FIRE - 
VEHICLE POSITION 


.0425 


1 


.0426 


.734 


.4037 


NO 


SURVIVABILITY - 
COORD OF RESERVE 


.0518 


1 


.0518 


.8959 


.3576 


NO 


VOLUME OF FIRE - 
COORD OF RESERVE 


. 0071 


1 


.0071 


. 123 


.7307 


NO 


VEHICLE POSITION 
- COORD OF RES 


.0979 


1 


.0979 


1.689 


. 1980 


NO 


RESIDUAL 


4.000 


69 


.0579 








TOTAL 


8.1420 


79 
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The survivability position parameter was the only main 
effect that did not demonstrate a statistically significant 
effect on mean values of M0P2A or M0P2B. The results for 
M0P2A indicated significant interaction among three of the 
parameters . 

Figures 17, 18 and 19 are plots of mean M0P2A values. 
Each plot depicts one of the significant two-way interactions. 
The format for these three plots is the same as discussed for 
the MOPl interaction. Figure 17 suggests that M0P2A decreases 
when the vehicle position parameter is changed to level two, 
and the decrease is more pronounced when survivability is at 
level two. 

Factor Means Plot- M0P2A 

Surv I vab I I I ty Pos 
■ ""At Level 1 ' At Level 2 



M0P2A M0P2A 




Levels' Vehicle Position 



Figure 17 'Vehicle Position and 
Survivability Position Interaction, M0P2A 
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Through inspection of the two terms (OPFOR Loss Rate and 
BLUEFOR Survival Rate) of M0P2A it appears that the 
emplacement of survivability positions can not overcome the 
BLUEFOR vulnerability introduced by the new vehicle positions. 
The fact that many BLUEFOR vehicles maneuver throughout the 
battle may have degraded the effectiveness of the selected 
survivability positions. 

Figure 18 reflects the interaction between the 

coordination of reserve and survivability position parameters. 
The effect of changing the coordination of reserve parameter 
to level two is smaller when the survivability parameter is at 
level two. Both terms of M0P2A decrease when the coordination 
of reserve parameter is changed to level two, but decreases in 
both terms are smaller when the survivability position 
parameter is at level two. With coordination of reserves at 
level two, it appears that adding these survivability 
positions allows the BLUEFOR to kill more OPFOR vehicles while 
suffering fewer losses. 
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Factor 


Means Plot' M0P2A 






Surv 1 vab 1 1 i ty Pos . 










“■“At Level 1 ' At Level 2 






M0P2A 




M0P2A 




1 . J 








1 . 3 


1 2 






- 


1 2 


1 , 1 ' 


^ 






1 . 1 






' ' — 
















1 


- 






1 


Q 9 


- 






Q 9 


0 . 8 








0 . 8 




1 




2 




Levels: Coordination of Reserve 





Figure 18 Survivability and Coordination 
of Reserve Interaction, M0P2A 



Figure 19 contains a plot showing interaction between 
vehicle position and coordination of reserve parameters for 
M0P2A. Holding both the coordination of the reseirve and the 
vehicle position parameters at level one results in a higher 
mean M0P2A value. In this combination both the OPFOR loss and 
the BLUEFOR survival rates are higher. Comparisons of the two 
terms in M0P2A indicate that lower BLUEFOR survival rates are 
associated with the coordination of the reserve parameter at 
level two. Under this interaction, holding the vehicle 
position parameter at level one may allow an earlier attrition 
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of OPFOR vehicles while presenting a lower risk to BLUEFOR 
vehicles involved in the reserve. 



Factor Means Plot: M0P2A 



Vehicle Position 
■ ■" At Level 1 — ' — At Level 2 



M0P2A M0P2A 




Levels: Coordination of Reserve 



Figure 19 Vehicle Position and 
Coordination of Reserve Interaction, 
M0P2A 



4. Model Adequacy; Residual Analysis 

Residual analyses included plotting the model's 
residuals against predicted MOP values and making a normal- 
normal plot of residuals against N(0,o^) random samples. The 
variance of the random samples was set equal to the variance 
of the residuals. The residual analyses supported the 
normality and homogeniety assumptions of our model. Figures 
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20 and 21 show scatter plots of residual values against 
predicted values for MOPl and MOP2A. 





RES 1 DUALS 


RES 1 DUAL 


PLOT FOR M0P1 






1 






' ! 




^ ^ ^ I 


I 
















- 




□ 


. 7 












- 














- 




□ 


, 7 












- 














- 




□ 


, 1 












- 






; 'I ' 








-□ 


, 2 




t • ? 1 




- 




. 














-□ 


, 5 


- 














- 










- 




-□ 


. 8 


- 


















. i 


, i , . , 


i , . . 


i 






2 . 


3 


2 , 5 


2.7 2 . 


m 

cn 


1 










PREDICTED 


VALUES OF I 


MOP1 





Figure 20 MOPl Residuals 
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RESIDUAL PLOT FOR M0P2A 
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Figure 21 MOP2A Residuals 



Figure 22 shows a cumulative probability plot of the residuals 
for MOPl. This plot is similar to cumulative probability 
plots for the other MOPs. The horizontal axis represents the 
residual values. The vertical axis is scaled so that the 
cumulative distribution function (cdf) of a normal 
distribution would plot along the solid line shown in the body 
of the graph. 
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Normal Probab i I i ty Plot 

cumulative percent 




Figure 22 Cumulative Probability Plot for MOPl 
Residuals 
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V CONCLUSIONS AND RECOMMENDATIONS 



A. SUMMARY AND CONCLUSIONS 

Progress was made in accomplishing each primary objective 
of this thesis. The analysis indicates that under certain 
conditions the output of the Janus (A) model is sensitive to 
changes in battle parameters. Janus (A) sensitivity was 
defined in terms of the changes in battle parameters that 
resulted in changes in mean Measure Of Performance (MOP) 
values. The analysis discussed in this thesis suggests that 
the amount of sensitivity demonstrated by Janus (A) is a 
function of two items: 

1. the Measure Of Performance (MOP) used as model output; 

and, 

2. the battle parameters (main effects) that are modified in 

a battle. 

The battle parameters in terms of BLUEFOR were, 

1. survivability positions; 

2. vehicle positions; 

3. volumes-of-f ire ; and, 

4. coordination of the reserve. 
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In this thesis Janus (A) did not demonstrate statistically 



significant sensitivity for output of the form 

BLUEFOR ROUNDS FIRED 

MOPl = 

OPFOR LOSSES 

The second form of model output considered in this thesis 
was M0P2 , 



OPFOR LOSSES ^ BLUEFOR SURVIVORS 
INITIAL OPFOR STRENGTH INITIAL BLUEFOR STRENGTH 

Janus (A) demonstrated statistically significant changes in 
M0P2 with respect to three of the four parameters. Only the 
survivability position parameter (as a main effect) failed to 
demonstrate a statistically significant effect on mean M0P2 
values . 

Janus (A) indicated the presence of interactions among the 
three of the four battle parameters with M0P2 as the model 
output. Volume-of-f ire was the only battle parameter that was 
not included in a significant two-way interaction. 

The sensitivity of Janus (A) output to changes in battle 
parameters demonstrates potential for using Janus (A) in the 
post-NTC training environment. A commander could use Janus (A) 
to help predict how tactical changes may affect battle 
outcome. The NTC provides results on how well the unit 
performed under the conditions present at the NTC. With a 
qualified battle implemented in Janus (A), the commander could 
alter the conditions that were present at the NTC and obtain 
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predictions of how these changes might have affected the 
battle. 

The results of this thesis may prove beneficial to TRAC- 
Monterey during phase III of the Janus and NTC Comparison 
Study. The indications of sensitivity in Janus (A) suggest the 
model could be a practical training asset. 

B. RECOMMENDATIONS 

Three recommendations are made concerning, 

1. the potential data collection effort at the NTC; 

2 . the need for further analysis concerning the sensitivity 
of the Janus (A) simulation; and, 

3. the further development of the NTC-Janus(A) training plan 
contained in the Appendix. 

If the use of the Janus (A) simulation increases, the 
training potential for its use with NTC scenarios should not 
be overlooked. To capitalize on this training potential, the 
personnel at the NTC should consider collecting battle data 
that can facilitate training with Janus (A). 

The sensitivity analysis in this thesis considered only 
one change in each of the four BLUEFOR parameters. With 
Janus (A) sensitivity demonstrated for only selected changes in 
these four parameters, it may prove helpful to investigate how 
sensitive Janus (A) output is to other changes in these and 
other parameters. Also, this thesis has demonstrated 
sensitivity for one Measure of Performance (MOP) with respect 
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to one NTC battle. It would be useful to investigate other 
MOPs and NTC battles. 

The Appendix contains an outline of a training plan for 
use by a unit after it has completed an NTC training rotation. 
While this outline is short of being a complete battalion 
plan, it might be considered a basis for developing complete 
plans. The plan contains a process based on the conclusions 
of this thesis and those in the TRAC-Monterey technical 
report. 
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APPENDIX: A PLAN FOR USING JANUS (A) IN POST-NTC ROTATION 
TRAINING 

A. PURPOSE 

The purpose of this plan is to aid the coiiunander in 
determining what effect changes to a plan might have had on 
the outcome of a previous NTC battle. The use of Janus (A) 
augments those actions normally taken by a commander to war 
game previous battles. 

B. SCOPE 

This plan pertains to NTC force-on-force (MILES- 
instrumented) missions executed at the NTC. This use of 
Janus (A) does not replace any method currently used to aid the 
commander in war gaming battles. Janus (A) should be used to 
help determine how changes in a plan affect battle outcome as 
measured by one or more Measures Of Performance (MOP's). 

This plan has three objectives: 

1. to provide a forum for the commander to review with 
subordinate leaders battles executed at the NTC; 

2 . to provide the commander with one estimate of the types 
of changes that may alter the outcome of a battle; and, 

3. to assist the commander in determining which changes in 
an operation could be expected to return the significant 
improvement in a MOP. 

Four conditions support the effective use of this process; 

1. the using unit must have access to and be familiar with 
the Janus (A) combat simulation; 
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2. the unit must have an understanding of the differences 
between the Janus (A) simulation and the force-on-force 
simulation conducted at the NTC (see TRAC Technical Report, 
TRAC-RDM-TR-191) ; 

3. an effective and timely qualification process must be 
established that allows a unit to conduct Janus (A) training 
soon after an NTC rotation; and, 

4. the using unit must have an understanding of the 
variability of Janus (A) output. 

The plan has seven steps. Figure 23 depicts these steps. 



NTC BATTLE REVIEW 

♦ 

SELECTION OF 
MEASURES OF PERFORMANCE 

I 

EXECUTION OF THE QUALIFIED 
NTC BATTLE IN JANUS 

♦ 

SELECTION OF PARAMETER CHANGES 
IN JANUS 

I 

APPLICATION OF CHANGES 

i 

ANALYSIS OF 

MEASURES OF PERFORMANCE 

I 

APPLICATION OF RESULTS 

Figure 23 Seven Steps in Post- 
NTC Training With Janus (A) 
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C. THE SEVEN-STEP PROCESS FOR TRAINING WITH JANUS (A) 

1. Step I. NTC Battle Review 

During Step I the coimnander and his staff review the 
mission and critical events of the NTC battle. This review is 
based upon the NTC Take Home Packages (THP) and the records 
kept by the unit. The outcome of this step should be an 
understanding of the battle with respect to, 

1. whether or not the mission was accomplished; 

2. the time and location of critical events that helped 

determine battle outcome; and, 

3. the identification of mission related goals. 

In determining if the mission was accomplished, the unit 
should consider the goals specified in the mission statement 
and the commander's concept of the operation. Particular 

attention should be given to goals that are in the form of, 

1. holding terrain; 

2. seizing terrain; 

3. destroying the OPFOR; 

4. preserving the BLUEFOR; and, 

5. maintaining contact with the OPFOR. 

Attention is directed to goals such as these since Janus (A) is 
suited to providing output related to attrition, location of 
forces and detection. The identified goals provide a basis 
for determining which events are critical. The critical 
events should be related to the achievement of the goals. 
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2. Step II. Selection of a Measure Of Performance (MOP) 
The outcome of this step is the selection of one or 

more Measures Of Performance (MOPs) . These MOPs are related 
to the mission, goals and timing of critical events identified 
in Step I. Two characteristics are important in a MOP; 

1. the MOP must be sensitive to changes made in the battle; 
and, 

2. the MOP must possess a close relationship to the unit's 
goals. 

A MOP that is based on attrition, engagement ranges or 
location of attrition should normally demonstrate sensitivity 
to changes in the battle. The following is an example of an 
attrition based MOP used in this thesis: 

OPFOR LOSSES BLUEFOR SURVIVORS 

MOP = + . 

INITIAL OPFOR STRENGTH INITIAL BLUEFOR STRENGTH 

3. Step III. Execution of the Qualified NTC Battle in 
Janus (A) 

The qualified NTC battle should be executed in 
Janus (A) without modification. The execution of the qualified 
battle serves several purposes. The qualified battle serves 
as a second review the battle. If need be, the critical 
events identified in Step I may be revised. 

This qualified battle serves as a reference point for the 
commander. The qualified battle results are compared to those 
results attained at the NTC. This comparison can give the 
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commander insights into whether the results of the battle are 
do to "task" performance or a combination of planning and task 
performance [Ref. 12]. Here the term "task" has a broad range 
of meanings. For example, task may refer to conducting a 
successful counter attack by the reserves or destroying 60% of 
the enemy in a specific engagement area. Consider the 
following example. A unit obtains results in a qualified 
scenario that are superior to those obtained under the same 
conditions at the NTC. This may indicate that the commander's 
plan (as executed at the NTC and in Janus) is sound but the 
performance of one or more important tasks at the NTC was 
poor. This poor task performance may later be attributed to 
conditions present at the NTC such as gunnery skills at levels 
below those portrayed in Janus. 

The qualified battle should be executed several times, so 
that variability in MOP values can be assessed. Once the 
commander is satisfied with his understanding of the qualified 
battle he is prepared for Step IV. 

4. Step IV. Selection of Changes to Parameters in 
Janus (A) 

The changes made to the qualified battle should be 
made with respect to, 

1. the BLUEFOR mission; 

2. the types of changes that were possible at the time of 
the battle given available time and resources; 
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3. the types of changes that might not have been possible at 
the time of the NTC battle but provide insight into those 
resources that may have improved the battle outcome had they 
been available (e-g. , greater engineer support, more 
artillery, additional ground forces) : and, 

4. the types of changes that can be executed in Janus (A). 

5. Step V. Application of The Changes 

The changes to the qualified scenario should be 
applied in a manner that allows attributing changes in the MOP 
means to each selected change in the battle. One method of 
applying changes to the battle is to apply each change 
separately and to hold other conditions as they were in the 
qualified battle. This "marginal design" does not take into 
account possible interactions between parameters. As 
discussed in this thesis interactions can be significant, and 
the potential for interactions should be considered during 
this process. Whatever method is used, particular attention 
should be given to the variability in Janus (A). Again, 
several Janus (A) runs should be conducted for each scenario 
that has been altered. 

6. Step VI. Analysis of the Measure Of Performance (MOP) 

Values 

Comparisons of MOP values are affected by, 

1. the inherent variability in battle outcome; and, 

2. the possible variability introduced by operator 
interactions during the execution of a Janus (A) scenario. 

The using unit should be aware of the inherent variability in 

Janus (A) outputs even with constant inputs. The variability 
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introduced by operator interaction should be minimized 
whenever possible. Minimizing operator interaction reduces 
the variability in MOP values that may be attributed to the 
operator attempting to replicate the same actions repeatedly. 
When possible, the changes should be applied so that the 
Janus (A) scenario can be executed without any operator 
interaction. If operator interaction is required, then the 
events that require initiation should be scripted. This 
scripting of events will help minimize the variability that 
may occur in the timing of operator controlled events. 

7. Step VII. Application of the Janus (A) Training Results 

Janus (A) training results can be applied in at least 
two areas. One area is in future planning and training. The 
Janus (A) results may indicate certain maneuvers or employment 
of systems that could increase a unit's ability to accomplish 
similar missions. If a tactical maneuver appeared 
constructive in the Janus (A) simulation, the commander might 
consider attempting a similar maneuver during subsequent 
training. If the commander has applied variations of a 
tactical plan in Janus (A), the simulation may provide 
indications of which variations seem the most promising for 
accomplishing the mission. 

A second area is identifying resources that may aid in 
a unit's ability to accomplish missions similar to that in the 
qualified scenario. For example, changes in engineer support 
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seen as useful in Janus (A) could suggest what 
additional assets could lead to improvements 
outcome . 



levels of 
in battle 
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